163 citations · 372 across the 24 of their papers we have counts for
3 papers · 2 filters
Dodo: Dynamic Contextual Compression for Decoder-only LMs
Guanghui Qin, Corby Rosset, Ethan C. Chau +2
Transformer-based language models (LMs) are inefficient in long contexts. We propose Dodo, a solution for context compression. Instead of one vector per token in a standard transfo…
Automatic Pair Construction for Contrastive Post-training
Canwen Xu, Corby Rosset, Ethan C. Chau +6
Alignment serves as an important step to steer large language models (LLMs) towards human preferences. In this paper, we propose an automatic way to construct contrastive data for…
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories
Suyu Ge, Chenyan Xiong, Corby Rosset +3
In this paper we improve the zero-shot generalization ability of language models via Mixture-Of-Memory Augmentation (MoMA), a mechanism that retrieves augmentation documents from m…