116 citations · 228 across the 20 of their papers we have counts for
22 papers
How to Loop Your Transformer: Loop Scaling, Specialization, and Parallel Inference
Chanakya Ekbote, Artidoro Pagnoni, Devendra Singh Sachan +5
Looping in transformer models reuses weights by repeatedly applying the same transformer, or parts of it, making the model deeper without increasing its parameter count. Whether lo…
Muse Glimmer 30B: An On-Device Agentic Model
Arash Ebrahim Abtahizadeh, Muna Aghamelu, Adil Ahmed +327
We introduce Muse Glimmer, an open-source 30B parameter causal language model with perception encoder, purpose-built for autonomous agentic tasks on consumer hardware. Muse Glimmer…
AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
Yun He, Wenzhe Li, Hejia Zhang +22
Recent progress in large language models (LLMs) has led to impressive performance on a range of tasks, yet advanced instruction following (IF)-especially for complex, multi-turn, a…
Explore Theory of Mind: Program-guided adversarial data generation for theory of mind reasoning
Melanie Sclar, Jane Yu, Maryam Fazel-Zarandi +4
Do large language models (LLMs) have theory of mind? A plethora of papers and benchmarks have been introduced to evaluate if current models have been able to develop this key abili…
Self-Consistency Preference Optimization
Archiki Prasad, Weizhe Yuan, Richard Yuanzhe Pang +6
Self-alignment, whereby models learn to improve themselves without human annotation, is a rapidly growing research area. However, existing techniques often fail to improve complex…
To the Globe (TTG): Towards Language-Driven Guaranteed Travel Planning
Da JU, Song Jiang, Andrew Cohen +8
Travel planning is a challenging and time-consuming task that aims to find an itinerary which satisfies multiple, interdependent constraints regarding flights, accommodations, attr…
Self-Taught Evaluators
Tianlu Wang, Ilia Kulikov, Olga Golovneva +7
Model-based evaluation is at the heart of successful model development -- as a reward model for training, and as a replacement for human evaluation. To train such evaluators, the s…
PathFinder: Guided Search over Multi-Step Reasoning Paths
Olga Golovneva, Sean O'Brien, Ramakanth Pasunuru +4
With recent advancements in large language models, methods like chain-of-thought prompting to elicit reasoning chains have been shown to improve results on reasoning tasks. However…
DOMINO: A Dual-System for Multi-step Visual Language Reasoning
Peifang Wang, Olga Golovneva, Armen Aghajanyan +4
Visual language reasoning requires a system to extract text or numbers from information-dense images like charts or plots and perform logical or arithmetic reasoning to arrive at a…
Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning
Lili Yu, Bowen Shi, Ramakanth Pasunuru +24
We present CM3Leon (pronounced "Chameleon"), a retrieval-augmented, token-based, decoder-only multi-modal language model capable of generating and infilling both text and images. C…
Shepherd: A Critic for Language Model Generation
Tianlu Wang, Ping Yu, Xiaoqing Ellen Tan +7
As large language models improve, there is increasing interest in techniques that leverage these models' capabilities to refine their own outputs. In this work, we introduce Shephe…
Scaling Speech Technology to 1,000+ Languages
Vineel Pratap, Andros Tjandra, Bowen Shi +13
Expanding the language coverage of speech technology has the potential to improve access to information for many more people. However, current speech technology is restricted to ab…
Cocktail HuBERT: Generalized Self-Supervised Pre-training for Mixture and Single-Source Speech
Maryam Fazel-Zarandi, Wei-Ning Hsu
Self-supervised learning leverages unlabeled data effectively, improving label efficiency and generalization to domains without labeled data. While recent work has studied generali…
ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning
Olga Golovneva, Moya Chen, Spencer Poff +4
Large language models show improved downstream task performance when prompted to generate step-by-step reasoning to justify their final answers. These reasoning steps greatly impro…
Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems
Yi-Lin Tuan, Sajjad Beygi, Maryam Fazel-Zarandi +3
Users interacting with voice assistants today need to phrase their requests in a very specific manner to elicit an appropriate response. This limits the user experience, and is par…
Logical Reasoning for Task Oriented Dialogue Systems
Sajjad Beygi, Maryam Fazel-Zarandi, Alessandra Cervone +2
In recent years, large pretrained models have been used in dialogue systems to improve successful task completion rates. However, lack of reasoning capabilities of dialogue platfor…
Alexa Conversations: An Extensible Data-driven Approach for Building Task-oriented Dialogue Systems
Anish Acharya, Suranjit Adhikari, Sanchit Agarwal +28
Traditional goal-oriented dialogue systems rely on various components such as natural language understanding, dialogue state tracking, policy learning and response generation. Trai…
Dialog Simulation with Realistic Variations for Training Goal-Oriented Conversational Systems
Chien-Wei Lin, Vincent Auvray, Daniel Elkind +10
Goal-oriented dialog systems enable users to complete specific goals like requesting information about a movie or booking a ticket. Typically the dialog system pipeline contains mu…
Data Augmentation for Training Dialog Models Robust to Speech Recognition Errors
Longshaokan Wang, Maryam Fazel-Zarandi, Aditya Tiwari +2
Speech-based virtual assistants, such as Amazon Alexa, Google assistant, and Apple Siri, typically convert users' audio signals to text data through automatic speech recognition (A…
Towards Personalized Dialog Policies for Conversational Skill Discovery
Maryam Fazel-Zarandi, Sampat Biswas, Ryan Summers +4
Many businesses and consumers are extending the capabilities of voice-based services such as Amazon Alexa, Google Home, Microsoft Cortana, and Apple Siri to create custom voice exp…
Investigation of Error Simulation Techniques for Learning Dialog Policies for Conversational Error Recovery
Maryam Fazel-Zarandi, Longshaokan Wang, Aditya Tiwari +1
Training dialog policies for speech-based virtual assistants requires a plethora of conversational data. The data collection phase is often expensive and time consuming due to huma…
Learning Robust Dialog Policies in Noisy Environments
Maryam Fazel-Zarandi, Shang-Wen Li, Jin Cao +4
Modern virtual personal assistants provide a convenient interface for completing daily tasks via voice commands. An important consideration for these assistants is the ability to r…