42 citations · 70 across the 13 of their papers we have counts for
14 papers
BIG-Bench Extra Hard
Mehran Kazemi, Bahare Fatemi, Hritik Bansal +17
Large language models (LLMs) are increasingly deployed in everyday applications, demanding robust general reasoning capabilities and diverse reasoning skillset. However, current LL…
Expert Routing with Synthetic Data for Continual Learning
Yewon Byun, Sanket Vaibhav Mehta, Saurabh Garg +4
In many real-world settings, regulations and economic incentives permit the sharing of models but not data across institutional boundaries. In such scenarios, practitioners might h…
Making Scalable Meta Learning Practical
Sang Keun Choe, Sanket Vaibhav Mehta, Hwijeen Ahn +4
Despite its flexibility to learn diverse inductive biases in machine learning programs, meta learning (i.e., learning to learn) has long been recognized to suffer from poor scalabi…
Regularizing Self-training for Unsupervised Domain Adaptation via Structural Constraints
Rajshekhar Das, Jonathan Francis, Sanket Vaibhav Mehta +3
Self-training based on pseudo-labels has emerged as a dominant approach for addressing conditional distribution shifts in unsupervised domain adaptation (UDA) for semantic segmenta…
DSI++: Updating Transformer Memory with New Documents
Sanket Vaibhav Mehta, Jai Gupta, Yi Tay +6
Differentiable Search Indices (DSIs) encode a corpus of documents in model parameters and use the same model to answer user queries directly. Despite the strong performance of DSI…
An Introduction to Lifelong Supervised Learning
Shagun Sodhani, Mojtaba Faramarzi, Sanket Vaibhav Mehta +4
This primer is an attempt to provide a detailed summary of the different facets of lifelong learning. We start with Chapter 2 which provides a high-level overview of lifelong learn…
Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models
Clara Na, Sanket Vaibhav Mehta, Emma Strubell
Model compression by way of parameter pruning, quantization, or distillation has recently gained popularity as an approach for reducing the computational requirements of modern dee…
An Empirical Investigation of the Role of Pre-training in Lifelong Learning
Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar +1
The lifelong learning paradigm in machine learning is an attractive alternative to the more prominent isolated learning scheme not only due to its resemblance to biological learnin…
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
Vamsi Aribandi, Yi Tay, Tal Schuster +11
Despite the recent success of multi-task learning and transfer learning for natural language processing (NLP), few works have systematically studied the effect of scaling up the nu…
Improving Compositional Generalization with Self-Training for Data-to-Text Generation
Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay +3
Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). Such representations are compositional and it is cost…
Efficient Meta Lifelong-Learning with Limited Memory
Zirui Wang, Sanket Vaibhav Mehta, Barnabás Póczos +1
Current natural language processing models work well on a single task, yet they often fail to continuously learn new tasks without forgetting previous ones as they are re-trained t…
Learning Rhyming Constraints using Structured Adversaries
Harsh Jhamtani, Sanket Vaibhav Mehta, Jaime Carbonell +1
Existing recurrent neural language models often fail to capture higher-level structure present in text: for example, rhyming patterns present in poetry. Much prior work on poetry g…
Spatial Co-location Pattern Mining - A new perspective using Graph Database
Sanket Vaibhav Mehta, Shagun Sodhani, Dhaval Patel
Spatial co-location pattern mining refers to the task of discovering the group of objects or events that co-occur at many places. Extracting these patterns from spatial data is ver…
Towards Semi-Supervised Learning for Deep Semantic Role Labeling
Sanket Vaibhav Mehta, Jay Yoon Lee, Jaime Carbonell
Neural models have shown several state-of-the-art performances on Semantic Role Labeling (SRL). However, the neural models require an immense amount of semantic-role corpora and ar…