178 citations · 549 across the 48 of their papers we have counts for
14 papers · 1 filter
On Improving Summarization Factual Consistency from Natural Language Feedback
Yixin Liu, Budhaditya Deb, Milagro Teruel +3
Despite the recent progress in language generation models, their outputs may not always meet user expectations. In this work, we study whether informational feedback in natural lan…
Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructions
Alexey Skrynnik, Zoya Volovikova, Marc-Alexandre Côté +9
The adoption of pre-trained language models to generate action plans for embodied agents is a promising research strategy. However, execution of instructions in real or simulated e…
Boosting Natural Language Generation from Instructions with Meta-Learning
Budhaditya Deb, Guoqing Zheng, Ahmed Hassan Awadallah
Recent work has shown that language models (LMs) trained with multi-task \textit{instructional learning} (MTIL) can solve diverse NLP tasks in zero- and few-shot settings with impr…
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
Ganesh Jawahar, Subhabrata Mukherjee, Xiaodong Liu +6
Mixture-of-Expert (MoE) models have obtained state-of-the-art performance in Neural Machine Translation (NMT) tasks. Existing works in MoE mostly consider a homogeneous design wher…
ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels
Yue Zhao, Guoqing Zheng, Subhabrata Mukherjee +2
Existing works on anomaly detection (AD) rely on clean labels from human annotators that are expensive to acquire in practice. In this work, we propose a method to leverage weak/no…
IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022
Julia Kiseleva, Alexey Skrynnik, Artem Zholus +14
Human intelligence has the remarkable ability to adapt to new tasks and environments quickly. Starting from a very young age, humans acquire new skills and learn how to solve new t…
Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021
Julia Kiseleva, Ziming Li, Mohammad Aliannejadi +18
Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new t…
Leveraging Locality in Abstractive Text Summarization
Yixin Liu, Ansong Ni, Linyong Nan +4
Neural attention models have achieved significant improvements on many natural language processing tasks. However, the quadratic memory complexity of the self-attention module with…
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee +4
Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of…
Pathologies of Pre-trained Language Models in Few-shot Fine-tuning
Hanjie Chen, Guoqing Zheng, Ahmed Hassan Awadallah +1
Although adapting pre-trained language models with few examples has shown promising performance on text classification, there is a lack of understanding of where the performance ga…
Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners
Shashank Gupta, Subhabrata Mukherjee, Krishan Subudhi +4
Traditional multi-task learning (MTL) methods use dense networks that use the same set of shared weights across several different tasks. This often creates interference where two o…
Knowledge Infused Decoding
Ruibo Liu, Guoqing Zheng, Shashank Gupta +5
Pre-trained language models (LMs) have been shown to memorize a substantial amount of knowledge from the pre-training corpora; however, they are still limited in recalling factuall…
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy
Tianlong Chen, Zhenyu Zhang, Yu Cheng +2
Vision transformers (ViTs) have gained increasing popularity as they are commonly believed to own higher modeling capacity and representation flexibility, than traditional convolut…
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata Mukherjee, Xiaodong Liu +5
Knowledge distillation (KD) methods compress large models into smaller students with manually-designed student architectures given pre-specified computational cost. This requires s…