4 papers
Train Separately, Merge Together: Modular Post-Training with Mixture-of-Experts
Jacob Morrison, Sanjay Adhikesaven, Akshita Bhagia +3
Extending a fully post-trained language model with new domain capabilities is fundamentally limited by monolithic training paradigms: retraining from scratch is expensive and scale…
Beyond One Output: Visualizing and Comparing Distributions of Language Model Generations
Emily Reif, Claire Yang, Jared Hwang +3
Users typically interact with and evaluate language models via single outputs, but each output is just one sample from a broad distribution of possible completions. This interactio…
Encode Once, Decode Never: Reusing Audio LM Internals for Efficient Temporal Localization
Joseph An, Phillip Keung, Jiaqi Wang +2
Audio language models process input audio into rich frame-level representations, but the standard approach to temporal localization generates timestamps as sequences of text tokens…
Summarization-Based Document IDs for Generative Retrieval with Language Models
Haoxin Li, Daniel Cheng, Phillip Keung +2
Generative retrieval (Wang et al., 2022; Tay et al., 2022) is a popular approach for end-to-end document retrieval that directly generates document identifiers given an input query…