178 citations · 549 across the 48 of their papers we have counts for
48 papers
Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
Dujian Ding, Ankur Mallick, Chi Wang +5
Large language models (LLMs) excel in most NLP tasks but also require expensive cloud servers for deployment due to their size, while smaller models that can be deployed on lower c…
Teaching Language Models to Hallucinate Less with Synthetic Tasks
Erik Jones, Hamid Palangi, Clarisse Simões +5
Large language models (LLMs) frequently hallucinate on abstractive summarization tasks such as document-based question-answering, meeting summarization, and clinical report generat…
EcoAssistant: Using LLM Assistant More Affordably and Accurately
Jieyu Zhang, Ranjay Krishna, Ahmed H. Awadallah +1
Today, users ask Large language models (LLMs) as assistants to answer queries that require external knowledge; they ask about the weather in a specific city, about stock prices, an…
Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation
Chen Dun, Mirian Hipolito Garcia, Guoqing Zheng +3
Large Language Models (LLMs) have the ability to solve a variety of tasks, such as text summarization and mathematical questions, just out of the box, but they are often trained wi…
Automatic Pair Construction for Contrastive Post-training
Canwen Xu, Corby Rosset, Ethan C. Chau +6
Alignment serves as an important step to steer large language models (LLMs) towards human preferences. In this paper, we propose an automatic way to construct contrastive data for…
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Qingyun Wu, Gagan Bansal, Jieyu Zhang +11
AutoGen is an open-source framework that allows developers to build LLM applications via multiple agents that can converse with each other to accomplish tasks. AutoGen agents are c…
SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
Luciano Del Corro, Allie Del Giorno, Sahaj Agarwal +3
Autoregressive large language models (LLMs) have made remarkable progress in various natural language generation tasks. However, they incur high computation cost and latency result…
Orca: Progressive Learning from Complex Explanation Traces of GPT-4
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar +3
Recent research has focused on enhancing the capability of smaller models through imitation learning, drawing on the outputs generated by large foundation models (LFMs). A number o…
Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings
Yehya Farhat, Hamza ElMokhtar Shili, Fangshuo Liao +7
Mixture-of-Experts (MoEs) achieve scalability by dynamically activating subsets of their components. Yet, understanding how expertise emerges through joint training of gating mecha…
GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions
Woojeong Jin, Subhabrata Mukherjee, Yu Cheng +5
Generalization to unseen tasks is an important ability for few-shot learners to achieve better zero-/few-shot performance on diverse tasks. However, such generalization to vision-l…
Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions
Shrestha Mohanty, Negar Arabzadeh, Julia Kiseleva +6
Human intelligence's adaptability is remarkable, allowing us to adjust to new tasks and multi-modal environments swiftly. This skill is evident from a young age as we acquire new a…
Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback
Nikhil Mehta, Milagro Teruel, Patricio Figueroa Sanz +3
Many approaches to Natural Language Processing (NLP) tasks often treat them as single-step problems, where an agent receives an instruction, executes it, and is evaluated based on…
Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference
Chi Wang, Susan Xueqing Liu, Ahmed H. Awadallah
Large Language Models (LLMs) have sparked significant interest in their generative capabilities, leading to the development of various commercial applications. The high cost of usi…
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models
Saghar Hosseini, Hamid Palangi, Ahmed Hassan Awadallah
Large-scale Pre-Trained Language Models (PTLMs) capture knowledge from massive human-written data which contains latent societal biases and toxic contents. In this paper, we levera…
On Improving Summarization Factual Consistency from Natural Language Feedback
Yixin Liu, Budhaditya Deb, Milagro Teruel +3
Despite the recent progress in language generation models, their outputs may not always meet user expectations. In this work, we study whether informational feedback in natural lan…
Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructions
Alexey Skrynnik, Zoya Volovikova, Marc-Alexandre Côté +9
The adoption of pre-trained language models to generate action plans for embodied agents is a promising research strategy. However, execution of instructions in real or simulated e…
Boosting Natural Language Generation from Instructions with Meta-Learning
Budhaditya Deb, Guoqing Zheng, Ahmed Hassan Awadallah
Recent work has shown that language models (LMs) trained with multi-task \textit{instructional learning} (MTIL) can solve diverse NLP tasks in zero- and few-shot settings with impr…
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
Ganesh Jawahar, Subhabrata Mukherjee, Xiaodong Liu +6
Mixture-of-Expert (MoE) models have obtained state-of-the-art performance in Neural Machine Translation (NMT) tasks. Existing works in MoE mostly consider a homogeneous design wher…
ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels
Yue Zhao, Guoqing Zheng, Subhabrata Mukherjee +2
Existing works on anomaly detection (AD) rely on clean labels from human annotators that are expensive to acquire in practice. In this work, we propose a method to leverage weak/no…
IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022
Julia Kiseleva, Alexey Skrynnik, Artem Zholus +14
Human intelligence has the remarkable ability to adapt to new tasks and environments quickly. Starting from a very young age, humans acquire new skills and learn how to solve new t…
Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021
Julia Kiseleva, Ziming Li, Mohammad Aliannejadi +18
Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new t…
Leveraging Locality in Abstractive Text Summarization
Yixin Liu, Ansong Ni, Linyong Nan +4
Neural attention models have achieved significant improvements on many natural language processing tasks. However, the quadratic memory complexity of the self-attention module with…
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee +4
Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of…
Pathologies of Pre-trained Language Models in Few-shot Fine-tuning
Hanjie Chen, Guoqing Zheng, Ahmed Hassan Awadallah +1
Although adapting pre-trained language models with few examples has shown promising performance on text classification, there is a lack of understanding of where the performance ga…